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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in 
the application: 

Listing of Claims: 

1. (Currently Amended) A method for producing a multimedia 
summary of G ummariaing at least one multimedia stream the summa ry comprising 
key elements selected from said at least one multimedia stream (101, 102) , the 
method comprising: 

a. ) one of receiving and retrieving said at least one multimedia stream 
(101, 102) comprising video, audio and text information; 

b. ) dividing the at least one multimedia stream (101, 102) into a video 
substream (SQ^, an audio sub-stream and a text sub-stream (30^; 

c. ) identifying for potential inclusion in the summary v ideo, audio and 
text key elements from said video (303), audio (305) and text (307) sub-streams, 
respectively; 

d. ) computing an importemce value for the identified video, audio and 
text key elements identified at said step (c); 

e. ) first filtering the identified video, audio and text key elements to 
exclude those key elements whose associated importance value is less than a pre- 
defined video, audio and text importance threshold, respectively; and 

f ) second filtering the remaining key elements from said step (e) in 
accordance with a user profile; 
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g. ) third filtering the remaining key elements fi-om said step (f) in 
accordance with network and user device constraints; and 

h. ) outputting a multimedia summary (120) which comprises from the 
key elements remaining from said step (g). 

2. (Currently Amended) The method of Claim 1, wherein said at least 
one multimedia stream (101, 102) is one of an analog and digital multimedia 
stream. 

3. (Currently Amended) The method of Claim 1, wherein the step of 
dividing the at least one multimedia stream (101, 102) into a video sub-stream (303) 
further comprises the step of identifying and grouping said at least one multimedia 
stream (101, 102) into a plurality of news stories (330) where each identified news 
story (330) is comprised of an anchor portion (311, 312) and a reportage (321,322) 
portion. 

4. (Currently Amended) The method of Claim 1, wherein the step of 
dividing the at least one multimedia stream (101, 102) into an audio sub-stream 
(305) fiirther comprises dividing said at least one multimedia stream (101, 102) into 
a plurality of equal-sized frames (30§) of a fixed time duration. 

5. (Currently Amended) The method of Claim 1, wherein the step of 
dividing the at least one multimedia stream (101, 102) into a text sub-stream (30^ 
further comprises dividing said at least one multimedia stream (101, 102) into a 
plurality of frames (308) wherein each frame of said plurality of frames is defined 
on a word boundary. 
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6. (Currently Amended) The method of Claim 1, wherein the act of 
identifying video, audio and text key elements from said video audio (S0§) 
and text (307) sub-streams further comprise the acts of: 

1. ) identifying low (S40), mid mO) and high level (940) features from 
the plurality of frames which comprise said video 4303^, audio (305) and text W?). 
substreams; 

2. ) determining an importance value to each of said extracted low 
(540), mid (740) and high level (040) features from said identifying act; 

3. ) computing a frame importance value for each of said plurality of 
frames which comprise said video (303), audio (305) and text (307) sub-streams as a 
function of the importance values of the feature importance values determined at 
said determining act; 

4. ) combining the frames into segments in each of said video (303), 
audio (305) and text (307) sub-streams; 

5. ) computing an importance value per segment for each segment from 
said combining act; 

6. ) ranking the segments based on said computed importance value at 
said computing step; and 

7. ) identifying key elements based on s£ud remked segments. 

7. (Currently Amended) The method of Claim 6, wherein said act (3) 
of computing a frame importance value for each of ssdd extracted low (510) , mid 
(710) and high level (040) features further comprises computing said importance 
value by one of deterministic, statistical and conditional probabiUty mesuis. 
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8. (Currently Amended) The method of Claim 7, wherein said 
prnbnhilintic conditional probability means comprises computing said frame 
importance value as one of a Gaussian, Poisson, Rayleigh and Bernoulli 
distribution. 

9. (Currently Amended) The method of Claim 8, farther comprising 
computing the w herein oaid Gaussian distribution according to the equation fef 
computing oaid frame importonco value io computed ao : 

where: 9 is any of the features; 

01 is the average of the feature veJuc; and 

02 is the expected deviation. 

10. (Currently Amended) The method of Claim 7, wherein said 
deterministic means comprises computing said frame importance value according to 
the equation a s: 

Frame Importance = X Wifi 

where: fi represent low, mid-level and high-level features; and 
Wi represent weighting factors for weighting said features. 

11. (Original) The method of Claim 6, wherein said step (4) of combining 

the frames into video segments further comprises combining said frames by one of 

family histogram computation means and shot change detection means. 
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12. (Currently Amended) The method of Claim 6, wherein said step (4) 
of combining the frames into audio segments further comprises the steps of: 

categorizing each frame from said audio sub-stream (305) as one of a speech 
frame, a music frame, a silence frame, a noise frame, a speech + speech frame, a 
speech + noise frame and a speech + music frame; and 

grouping consecutive frames having the same categorization. 

13. (Currently Amended) The method of Claim 6, wherein act step (4) 
of combining the frames into text segments further comprises combining said 
frames based on punctuation included in said text sub-stre£im (307) . 

14. (Original) The method of Claim 6, wherein said step (5) of 
computing an importance value per segment further comprises averaging the franie 
importance values for those frames which comprise said segment. 

15. (Original) The method of Claim 6, wherein said step (5) of 
computing an importance value per segment further comprises using the highest 
frame importance value in said segment. 

16. (Original) The method of Claim 6, wherein said step (7) of 
identifying key elements based on said rankings further comprises identifying key 
elements whose segment ranking exceeds a predetermined segment ranking 
threshold. 
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17. (Original) The method of Claim 6, wherein said step (7) of 
identifying key elements based on said rankings further comprises identifying key 
elements whose segment ranking both exceeds a predetermined segment ranking 
threshold and constitute a local maxima. 

18. (Original) The method of Claim 6, wherein said step (7) of 
identifying key elements based on said rankings further comprises identifying key 
elements whose segment ranking constitutes a local maxima. 

19. (Currently Amended) A system (iOO) for producing a multimedia 
summary of oummorizing at least one multimedia stream (101, 102) , the summary 
comprising key elements selected from said at last one multimedia stream, the 
system comprising: a modality recognition and diyision (MRAD) module (103) 
comprising a story segment identifier (SSI) module (103a) , an audio identifier (Al) 
module (103b) and a text identifier (TI) module OOSe), the MRAD module 003) 
communicatiyely coupled to a first external source (440) for receiying said at least 
one multimedia stream (101, 102) , the MRAD module (403) communicatively 
coupled to a second external source (442) for receiving said at least one multimedia 
stream (101, 102) , the MRAD module (403) dividing said at least one multimedia 
stream (101, 102) into a video (303) , an audio (30§) and a text (30?) sub-stream and 
outputting said video (303), audio (305) and text (30?) sub-streams to a key element 
identifier ( KEI) module (405), the KEI module (405) comprising a feature extraction 
(FE) module (407) and an importance value (IV) module (409) for identifying key 
elements fi-om within said video (303), audio (305) and text (307) sub-streams and 
assigning importance values thereto, the KEI module (405) communicatively 
coupled to a key element filter (KEF) (444) for receiving the identified key elements 
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and filtering said key elements that exceed a pre-determined threshold criteria, the 
KEF module (Mi^ communicatively coupled to a user profile filter (UPF) (443^ for 
receiving filtered key elements and fiirther filtering said filtered key elements in 
accordance with a user profile, the UPF module (ii3^ communicatively coupled to a 
network and device constraint (NADC) module ^i5), said NADC module tfi§) 
receiving said fiirther filtered key elements and fiarther filtering said fiirther 
filtered key elements in accordance with network and/or user device constraints, the 
NADC module (ii5) outputting a multimedia summary (120) of said at least one 
multimedia stream which comprises kev elements remaining afl;er said fiirther 
filtering in the NADC module (101. 102) . 

20. (Currently Amended) The system of Claim 19, further comprising a 
user preference database ^iii^ communicatively coupled to said UPF module (iiS) 
for storing user profiles. 

21. (Currently Amended) The system of Claim 19, wherein the first 
external source (iiO) is a broadcast channel selector. 

22. (Currently Amended) The system of Claim 19, wherein the first 
external source (110) is a video streaming source. 

23. (Currently Amended) The system of Claim 19, wherein said at 
least one multimedia stream (101, 102) is one of an analog and digital multimedia 
stream. 
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24. (Currently Amended) The system of Claim 19, wherein the NADC 
module is communicatively connected to an external network (i3S) coupled to 
a user device (124) . 

25. (Currently Amended) The system of Claim 19, wherein the 
network (122) is the Internet. 

26. (Currently Amended) An article of manufacture for producing a 
multimedia summary of summarizing at least one multimedia stream (101, 102) , 
the summary comprising kev elements selected from said at least one multimedia 
stream, the article comprising; a computer readable medium having computer 
readable code means embodied thereon, said computer readable program code 
means comprising: 

an act of one of receiving and retrieving said at least one multimedia 
stream (101, 102) comprising video, audio and text information; 

an act of dividing said at least one multimedia stream (101, 102) into a 
video sub-stream (303) , an audio sub-stream (305) and a text sub-stream (307) ; 

an act of identifying for potential inclusion in the summary v ideo, 
audio and text key elements from said video (303) , audio (305) and text (S059 sub- 
streams, respectively; 

an act of computing an importance value for the identified video, audio 
and text key elements identified at said identification act; 

an act of first filtering the identified video, audio and text key elements 
to exclude those key elements whose associated importance value is less than a pre- 
defined video, audio and text importance threshold, respectively; and 

an act of second filtering the remgdning key elements firom said first 
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filtering act in accordance with a user profile; 

an act of third filtering the remaining key elements fi-om said second 
filtering act in accordance with network and user device constraints; and 

an act of outputting a multimedia summary which comprises ^ 130^ 
from tho key elements remaining fi-om said third filtering act. 

27. (Currently Amended) The article of manufacture of Claim 26 
further wherein the act of identifying video, audio and text key elements from said 
video (30^, audio (306) and text (307) sub-30 streams, respectively, further 
comprises: 

an act of identifying low (SiO), mid and high level (©iO) features 
from the plurality of frames which comprise said video (303) , audio (30^ and text 
(30?) sub-streams; 

an act of determining an importance value to each of said extracted low 
(§iO), mid (TiO) and high level (840) features from said identifying act; 

an act of computing a frame importance value for each of said plurality 
of frames which comprise said video (303), audio (30§) and text (30^ sub-streams as 
a function of the importance values of the feature importance values determined at 
said determining step; 

an act of combining the frames into segments in each of said video 
(303), audio (305) and text (307) sub-streams; 

an act of computing an importance value per segment for each segment 
from said combining act; 

an act of ranking the segments based on said computed importance 
value at said computing act; and 

an act of identifying key elements based on said ranked segments. 
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